Deriving Ground Truth Labels for Regression Problems Using Annotator Precision

When training machine learning models with practical applications, a quality ground truth dataset is critical. Unlike in classification problems, there currently no effective method for determining single value or landmark from set of annotations regression problems. We propose novel deriving labels problems that considers the performance and precision individual annotators when identifying each label separately. In contrast to commonly accepted computing global mean, our does not assume annotator be equally capable completing specified task, but rather ensures higher-performing have greater contribution final result. The selection described within this paper provides means improving input data model development by removing lower-quality labels. study, we objectively demonstrate improved applying simulated where canonical position can known, as well sample collected crowd-sourced